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Abstract 


Given the ongoing alarm regarding uncontrollable costs of higher education, it would be 
reasonable to expect not only concern about the impact of MOOCs on educational 
outcomes, but also systematic efforts to document the resources expended on their 
development and delivery. However, there is little publicly available information on 
MOOC costs that is based on rigorous analysis. In this article, we first address what 
institutional resources are required for the development and delivery of MOOCs, based 
on interviews conducted with 83 administrators, faculty members, researchers, and 
other actors in the MOOCspace. Subsequently, we use the ingredients method to 
present cost analyses of MOOC production and delivery at four institutions. We find 
costs ranging from $38,980 to $325,330 per MOOC, and costs per completer of $74- 
$272, substantially lower than costs per completer of regular online courses, by merit of 
scalability. Based on this metric, MOOCs appear more cost-effective than online 
courses, but we recommend judging MOOCs by impact on learning and caution that 
they may only be cost-effective for the most self-motivated learners. By demonstrating 
the methods of cost analysis as applied to MOOCs, we hope that future assessments of 
the value of MOOCs will combine both cost information and effectiveness data to yield 
cost-effectiveness ratios that can be compared with the cost-effectiveness of alternative 
modes of education delivery. Such information will help decision-makers in higher 
education make rational decisions regarding the most productive use of limited 
educational resources, to the benefit of both learners and taxpayers. 
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Introduction 


At least since the 1990s concerns have arisen over the increasing costs and decreasing 
productivity of higher education, with technology-based reforms being promoted as a 
solution for institutions of higher education (IHEs) struggling to educate larger 
numbers of students with a wider range of incoming preparation and learning styles 
(e.g., Twigg, 1992; Rumble, 1997; Bowen, 2012, 2013; Barber, Donnelly, & Rizvi, 2013). 
Established IHEs have generally been slow to take advantage of technology to improve 
productivity in the delivery of education (Miller, 2010), for reasons that are more often 
“psychological, political, and cultural” rather than “conceptual, technical, or economic” 
(Dede, Ed., 2013, p. 52). However, a few pioneering institutions and numerous 
newcomers have gained traction swiftly by offering online or blended learning 
opportunities to both typical college-aged students and to older, non-traditional 
learners. As early as the 1970s, the Open University in the United Kingdom was able to 
offer distance education courses at a large enough scale to render institutional costs per 
student below the costs of similar courses at traditional campuses (Laidlaw & Layard, 
1974)- 

There is, however, limited evidence regarding the costs of technology-mediated distance 
instruction and mixed evidence as to whether it lowers the overall costs of education or 
increases them. Lack (2013) observes that inattention to costs is pervasive in 
postsecondary education, and highlights one of the few exceptions in the field of 
postsecondary online learning, the National Center for Academic Transformation 
(NCAT), which, according to its website, helps institutions use “information technology 
to re-design learning environments to produce better learning outcomes for students at 
a reduced cost to the institution.” Miller (2010) reports cost savings of 1396-77% across 
fifty instances of NCAT-supported course re-designs. Costs per student averaged $196 
across the fifty original, traditional versions of the courses while the versions that were 
re-designed with technology components averaged 39% less, at $119 per student. In one 
example, costs per student for a fine arts course offered by Florida Gulf Coast University 
dropped from $132 to $70 after it was transformed from an on-campus course into a 
fully online course. It is not clear, however, what method was used to establish costs or 
which personnel and other resources were included in the cost calculations. Twigg 
(2003) acknowledges that the NCAT estimates do not include costs of course 
development and transition from traditional to re-designed version, but she also argues 
that they do not reflect savings that can be achieved by increasing retention, reducing 
space utilization, or eliminating similar courses. 

Cota, Jayaram, and Laboissiere (2011) assert that the most productive colleges in the 
United States (U.S.), as defined by cost per degree (institution’s total annual costs 
divided by the number of degrees awarded) achieve their efficiencies through five 
strategies, one of which is keeping costs under control by re-designing instruction, often 
using technology to deliver some or all content and instruction at distance. On the other 
hand, Means, Bakia, and Murphy (2014) assert that online learning incurs greater 
investment costs than conventional instruction for program design, curriculum 
development, and development or selection of digital resources. Given the high fixed 
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costs of development of online instruction, and of technology-mediated distance 
education more generally, many experts argue that scale is essential to reducing costs 
per student (e.g., Boeke, Ed., 2001; Jones, 2004). Massive open online courses 
(MOOCs) would appear to offer the ideal opportunity to take advantage of scale given 
their potentially enormous enrollments. 

Online enrollment in the U.S. has grown at a rate between 6.1% and 36.5% in each year 
since 2002 (Allen & Seaman, 2013, 2014), and over the past two years MOOCs have 
begun to play a noticeable role in this growth. In 2013, 5% of 2,831 IHEs responding to 
Allen and Seaman’s (2014) annual survey about online learning were offering a MOOC, 
9% were planning to do so, and 53% were undecided as to whether to engage in this 
innovation. While it is clear that MOOCs have “... nudged almost every university 
toward developing an Internet strategy” (Lewin, 2013), there is little evidence that 
MOOCs have, as yet, contributed to lowering the costs of higher education. 

Given the continuing alarm regarding uncontrollable costs of higher education (e.g., 
Bowen, 2013; Kelly & Carey, Eds., 2013), it would be reasonable to expect not only 
concern about the impact of MOOCs on educational outcomes, but also systematic 
efforts to document the resources expended on their development and delivery. 
However, beyond the approximate estimates offered by Boddy et al. (2013), there is 
little publicly available information on MOOC costs that is based on rigorous analysis. 
Ithaka S+R (2014) documents hours spent by personnel in developing and delivering 
hybrid courses at the University System of Maryland, some of which integrated MOOCs 
or MOOC components, but does not translate these into costs. 

Moreover, it appears that lowering costs is not the highest priority for MOOC initiatives: 
among the 140 or so IHEs offering MOOCs in Allen and Seaman’s (2014) sample, less 
than ten indicated that exploring cost reductions was an objective for their MOOC 
initiatives. Hollands and Tirthali (2014) found that, of 29 institutions offering MOOCs, 
improving economics was a goal for only 38%. A recent poll by the Alliance for Higher 
Education and Democracy (AHEAD) at the University of Pennsylvania found that, 
among the approximately 44 respondents at institutions offering a MOOC, only 19% 
strongly agreed that MOOCs may be an effective mechanism for reducing costs of higher 
education (AHEAD, 2014). Goals that were as or more important than reducing costs to 
the IHEs in these studies included: increasing access to education, raising institutional 
visibility or building brand, increasing student recruitment, and improving or 
innovating pedagogy. 

Ruth (2013) explores the question of whether MOOCs can be used to help reduce college 
tuition and concludes that MOOCs may only contribute to lowering costs of higher 
education if combined with a reduction in labor costs, as experienced in successful 
implementations of NCAT’s course re-design model. Hoxby (2014) assesses the 
economic value of MOOCs and questions the assumption that cost reductions, via 
economies of scale, will be realized through MOOCs because she expects that the most 
popular MOOC instructors will eventually need to be paid high salaries. It is perplexing 
that MOOCs have taken hold without much evidence as to whether they are effective in 
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improving participant skills and knowledge, and without a firmer idea of their economic 
value, resource requirements, and costs. As Means et al. (2014) observe, “Both irrational 
exuberance and deep-seated fear concerning online learning are running high” (p. 42). 
If decision-makers are to make rational decisions about engaging in MOOC production, 
it is critical to know whether MOOCs are both effective and cost-effective in delivering 
quality education or related outcomes. 

In this article, our objectives are to address what institutional resources are required for 
the development and delivery of MOOCs, what are the associated costs per MOOC and, 
where the data are available, what is the cost per MOOC completer. We compare these 
findings with costs of other online and distance learning to assess whether MOOCs can 
deliver education more inexpensively at scale than alternative options. We hope that by 
demonstrating the methods of cost analysis as applied to MOOCs, future assessments of 
the value of MOOCs and other distance learning courses will combine both cost 
information and effectiveness data to yield cost-effectiveness ratios that can be 
compared with the cost-effectiveness of alternative modes of education delivery. Such 
information will help decision-makers in higher education make rational decisions 
regarding the most productive use of limited educational resources, to the benefit of 
both learners and taxpayers. 


Methods 


To elicit information regarding the resources required to develop and deliver MOOCs, 
we conducted a qualitative study (see Merriam, 2009) similar to that employed by 
Bacow, Bowen, Guthrie, Lack, and Long (2012) in their investigation of barriers to 
online learning in higher education. We interviewed 83 individuals across 62 public and 
private organizations including IHEs, research organizations, online learning platform 
providers, other for-profit education companies, and several additional stakeholders in 
the online learning space. Table 1 indicates the distribution of interviewees across 
institutional type. Thirty of our interviewees were administrators at IHEs, 22 were 
faculty members, 16 were executives at other institutions, 13 were researchers, one was 
an educational technologist, and one was a program officer at a foundation. 

Interviewees were identified by reviewing the academic and journalistic literature on 
MOOCs, the names of presenters and panelists at conferences on MOOCs or online 
learning in higher education, and the MOOC activities of institutions on the Internet. 
Many of our interviewees suggested other people for us to interview either at their own 
institutions or elsewhere. We contacted by e-mail individuals who appeared to be 
knowledgeable about MOOCs or online learning based on their position in deciding 
whether and how to engage with MOOCs, experience teaching or planning MOOCs, or 
relevant research and publications. 

We contacted too individuals on a rolling basis at 66 different institutions, 39 of which 
were IHEs. Most interviewees were based in the U.S., two were in China, two in the 
United Kingdom, and several were in Canada. Interviews were conducted between June 
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2013 and February 2014 and follow-ups by e-mail with interviewees to obtain updates 
and to verily information continued until May 2014. Almost half of the interviews were 
conducted face-to-face with the remainder conducted by telephone or Skype. Interviews 
averaged 75 minutes in length and followed a semi-structured interview protocol (see 
Merriam, 2009). Most interviewees agreed to be recorded, and the digital audio-files 
were subsequently transcribed. All interview notes and transcriptions were coded 
(LeCompte & Schensul, 1999) in NVivo software using themes initially derived from the 
interview protocol and iteratively refined as more granular topics were identified. 

Table 1 

Institutional Affiliations of Interviewees 


Type of institution 

Number of institutions 
represented* 

Number of 
interviewees 

Public universities 

16 

20 

Private universities 

14 

26 

Community colleges 

9 

10 

Platform providers 

5 

6 

Research organizations 

7 

8 

Other for-profit education companies 

4 

5 

Other institutions** 

7 

8 

Total 

62 

83 


* One person was interviewed at most institutions, but at a few institutions several individuals 
were interviewed, for example, to include administrators, faculty members, and researchers. 
**Other institutions: museum, K-12 school district, educational technology advocacy group, 
higher education association, venture capital firm, private foundation, independent consultant. 


Cost analyses were conducted using the ingredients method (Levin & McEwan, 2001) to 
estimate the costs of MOOC production and delivery at four of the institutions where we 
were able to obtain adequate data on resource use. We estimated costs for one 
connectivist MOOC (cMOOC) and seven xMOOCs. We focused on estimating personnel 
costs and assumed these would represent 75% of total costs, based on Levin and 
McEwan’s assertion that personnel costs typically account for 70-80% of total costs of 
educational interventions (see p. 53). We do not estimate costs individually for facilities, 
other equipment, and overhead but assume they amount to 25% of total costs. To 
estimate personnel costs we asked our interviewees detailed questions regarding role, 
qualifications, and hours spent by each person involved in MOOC development and 
production. In two cases, detailed records of time spent were collected by the 
institutions as part of their regular project management process. In one case, the MOOC 
instructor logged time spent on the MOOC on a daily basis and we obtained other 
personnel hours by interviewing the relevant individuals shortly after the conclusion of 
the MOOC. In the case of the cMOOC, we obtained retrospective estimates of hours 
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spent from the two instructors involved. We expect greatest accuracy when time spent is 
logged on a regular basis. 

In order to assign costs to personnel time, we used national average U.S. salaries for 
individuals in each relevant job category, as opposed to using actual salary levels of 
personnel at each specific institution, except in one case where some of the personnel 
costs were given to us directly. This approach not only respects the privacy of the 
individuals involved, but, more practically, allows for a comparison of the costs across a 
number of institutions without introducing local pricing influences. National average 
prices and benefits rates were obtained from the CBCSE Database of Educational 
Resource Prices which relies on multiple national surveys such as the National 
Compensation Survey, U.S. Department of Labor. Cost calculations were executed using 
the CBCSE Cost Tool Kit, an Excel-based application designed for the purpose of 
estimating costs of educational programs. 


Findings 


Resource Requirements for Developing and Delivering MOOCs 

We first review the resources required to produce and deliver MOOCs based on 
information provided by our interviewees. Subsequently, we present our estimates of 
the costs of MOOCs from the perspective of the producer (i.e., the college, university, or 
museum, as opposed to the platform provider or participant). We note that for MOOCs 
that are delivered via third-party platforms, there are often significant, additional costs 
to the platform provider which may be passed on to the MOOC producers through a 
direct charge for the platform services or a revenue-sharing agreement (see Young, 
2012; Kolowich, 2013). 

The major cost drivers we identified in MOOC production and delivery were: the 
number of faculty members, administrators, and instructional support personnel 
participating in the process; the quality of videography; the nature of the delivery 
platform; programming for special features such as computer code auto-graders, virtual 
labs, simulations, or gamification; analysis of platform data; and technical support for 
participants. MOOC production teams that were described to us seldom included fewer 
than five professionals and, in at least one instance, over 30 people were involved. 

All interviewees who had been involved in the development of a MOOC reported the 
effort being two to three times greater than creating a traditional course. These reports 
comport with written accounts such as Cima’s (2013). Instructors typically spent several 
hundred hours over several months preparing and re-purposing course materials, and 
practicing lecture delivery prior to video-taping; several days on actual shoots; and one 
to two days reviewing the finished video. To create one hour’s worth of MOOC video¬ 
lecture required three to ten hours of preparation according to several faculty members, 
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the lower end of the range being in instances where the materials were being re¬ 
purposed from existing lectures. To create ten minutes of voice-over-PowerPoint video 
required six to eight hours according to an interviewee at a private university. 

Development of MOOCs was deemed to be more time-consuming compared to 
traditional online courses due to MOOC-specific components such as high quality video, 
quizzes to substitute instructor-graded assignments, and peer-to-peer learning 
technologies. Several interviewees noted that the level of “polish” required for content 
and delivery was far greater than for traditional on-campus or online courses because of 
the more public nature of the MOOC. A number of interviewees likened the effort 
involved in creating a MOOC with writing a textbook in a team. At some institutions 
faculty members were granted a course release and/or paid stipends ranging from 
$3,ooo-$i5,ooo for developing and delivering a MOOC, but the opportunity costs of the 
instructor’s time are likely to be higher in many instances. We frequently heard 
estimates in the order of 400 hours of faculty member time per MOOC developed, the 
equivalent of 26% of an academic year. 

In addition to the direct costs of producing and delivering MOOCs, many of our 
interviewees provided insights into a plethora of additional considerations for 
institutions engaging with MOOCs. For example, MOOCs can only attract massive 
audiences if they are sufficiently marketed. While the platform providers such as 
Coursera, edX, and Academic Partnerships fulfill these marketing and communications 
functions for their partner institutions, those institutions using more “do-it-yourself’ 
platforms must find suitable advertising channels. Computing and Internet services for 
on-campus students participating in MOOCs may need to be increased or upgraded, for 
example, help desk support and retrofitting buildings to provide enough bandwidth 
capacity for many students to simultaneously stream or download video. Institutional 
websites and learning management systems need to provide an access point to relevant 
MOOCs. Cheal (2012) documents many of these issues as encountered by San Jose State 
University’s MOOC initiatives. 

A variety of administrative offices are likely to be involved in activities such as obtaining 
copyright permissions and establishing contracts between the institution and online 
platform provider, and between the institution and its faculty members to address 
intellectual property rights, revenue sharing, faculty compensation and workload issues. 
Compliance with disability regulations in MOOCs must be regularly audited and 
enforced, and accommodations made, for example, extra time on quizzes and exams for 
students with learning disabilities. For institutions providing credit for MOOCs, the 
student admissions, registration, billing, authentication, and crediting systems need to 
be aligned with platform enrollment procedures. If prerequisites are required for credit¬ 
earning participation in a course, a system must be developed to handle large numbers 
of students. 
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Costs of MOOC Production and Delivery 

Based on the cost analyses we conducted of MOOC production and delivery, we 
estimated personnel costs ranging between $29,000 and $244,000 per MOOC, 
depending on the number of people involved in the process, the amount of time 
dedicated, and the quality of video production. The costs of the platform, captioning, 
content hosting, and analysis of user data to populate the data dashboard were assumed 
by Coursera for all xMOOCs we analyzed. We estimate total costs per MOOC, including 
facilities, equipment, and overhead, of $38,980 to $325,330 (see Table 2). In two cases 
where course completion data were available, we present a cost per completer. Details of 
each institution’s MOOC(s) and our related cost analysis are presented below. 

Table 2 

Estimated Costs of MOOC Production and Delivery at Four Institutions 


Institution 

Type of 
MOOC 

Length of 
MOOC (weeks) 

Total estimated 
costs per MOOC 

Costs per 
completer 

Teachers College, 
Columbia University 

xMOOC 

8 

$38,980 

$74 

University of 

Manitoba 

cMOOC 

12 

$65,800 - 
$71,800 


American Museum 
of Natural History 

xMOOC 

4 

$104,620 

$272 

Large Midwestern 
university 

xMOOC 

5-8 

$203,770 - 

$ 325,330 

- 


Cost analysis for development and delivery of Connectivism and 
Connected Knowledge (a cMOOC). 

Connectivism and Connected Knowledge (CCK08), the first course to be dubbed a 
“MOOC,” was developed and delivered in 2008 by George Siemens and Stephen 
Downes. The 12-week course was offered at the University of Manitoba to 25 enrolled 
students for fee and for credit and also as a free, non-credit-bearing course to 2,300 
other participants (Downes, 2008). The course has been re-run three times since. 

Siemens estimated the time burden for CCK08 development and delivery as follows: 
100-150 hours on course design and development over a two month period; 70 hours 
per week on course delivery for the first two to three weeks (interacting with students 
and posting on discussion forums or writing blog posts to summarize discussion and 
activities), tapering down to 30 hours per week in the twelfth week. At the lower end of 
Siemens’ estimates, the total number of hours amount to 715. At the high end, they 
amount to 770. We estimate costs at each end of the range. 

Downes estimated his total time commitment for CCK08 at 88-108 hours: 20-40 hours 
in programming time to make adjustments to the gRSShopper course aggregation 
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software that he had developed over many years; 20 hours setting up the course 
website; and four hours per week during course delivery to maintain the site and 
prepare audio archives. No technology support personnel, learning designers, or 
teaching assistants (TAs) were utilized in the development and delivery of CCK08. 

Using U.S. national average salary and benefits rates for public postsecondary faculty 
members and public sector research scientists, the costs of personnel time to replicate 
CCK08 ranges from $49,400 to $53,800 and we estimate the total costs of between 
$65,800 and $71,790, as shown in Table 3. 

Re-runs of CCK08 required less design and development time. Additionally, with better 
course management software, weekly delivery time for the 2012 delivery fell to 30-40 
hours per week for the first two to three weeks. Some repeat students self-selected as 
TAs and reduced the instructors’ time burden by helping manage the forums, 
responding to inquiries, and providing guidance to new students. Set-up time for the 
course website dropped from 20 hours to four hours. For Siemens, we estimate the total 
time commitment for a CCK08 re-run at 284 hours: 20 hours to “refresh” the course 
design and resources before a new launch; 28 hours per week in delivery for the first 
three weeks; and 20 hours per week in delivery for the remaining nine weeks. For 
Downes, we estimate the total time commitment for a CCK08 re-run at 72 hours: four 
hours for website set-up; 20 hours to adjust gRSShopper to accommodate new tools; 
and four hours per week to maintain the course site. The possible range of time 
committed by the self-selected TAs could be very wide. We use an estimate of 350 hours 
total, under the assumption that the TAs collectively replace the reduced hours in 
Siemens’ delivery time. Total estimated costs for the re-run are $40,740, 38% lower 
than the low estimate for the first run. 

Table 3 

Estimated Costs for the First Run ofCCKo8 and Re-run 


Ingredient 

First run 
low estimate 

First run 
high estimate 

Re-run 

estimate 

Instructor 1 

$44,756 

$48,199 

$17,777 

Instructor 2 

$4,597 

$5,642 

$3,76i 

TAs 

- 

- 

$9,015 

Facilities, equipment, 
overhead 

$16,451 

$17,947 

$10,184 

Total 

$65,804 

$71,788 

S4Q,737 


Sources: George Siemens, formerly at Athabasca University, and Stephen Downes, National 
Research Council. 
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Costs of xMOOC production at a large midwestern university. 

Before its recent entry to the MOOCspace, this university, which requested partial 
anonymity, had already established an infrastructure for the development of online 
courses. In 2013, a small number of faculty members were invited to develop and deliver 
five- to eight-week MOOCs, primarily to showcase the university and engage new 
audiences. Each faculty member was assigned a design and support team of five to six 
people to help in the design and production of the MOOC, including a project manager, 
instructional designers, instructional technologists, and a liaison to the video 
production team. Additional personnel supervised the design and support teams, and 
provided programming capacity, overall project management, evaluation, and 
administrative services. 

As a routine part of the project management function at this university, detailed time 
logs are kept by each design team member so that costs for these personnel can be 
tracked accurately. We used the cost estimates provided by the university for these 
personnel in our analysis because we did not obtain enough detail regarding these 
personnel ingredients (e.g., specific role, level of experience, highest degree of 
education) to allow us to assign prices ourselves. Faculty member and TA time were not 
logged but we obtained estimates either during or after MOOC production and assigned 
relevant costs ourselves, using national average salary and benefits rates for 
postsecondary public institutions. For the first three MOOCs created and delivered, the 
hours spent per MOOC by various personnel were as follows: 200-500 hours for the 
MOOC design team, 700-900 hours for the video production team, 150-155 hours for 
technical support, 90-220 hours for the faculty member, and 650 hours for a TA in one 
MOOC. Total personnel hours were 1,140 for the least time-intensive MOOC and 2,245 
for the most demanding MOOC. The resulting cost estimates are shown in Table 4. 

Table 4 

Range Of Estimated Personnel Costs Per MOOC for Design, Production, and Delivery 
at a Large Midwestern University 


Type of personnel 

Low 

High 

Design and support team 

$70,000 

$125,000 

Computer programming unit 

$0 

$15,000 

Management (avg. across 3 MOOCs) 

$77,200 

$77,200 

Faculty member 

$5,630 

$13,770 

TA 

$0 

$13,029 

Total personnel costs 

$ 152,830 

$244,000 


Source: Evaluator at a large midwestern university. 


The faculty time burden was relatively low because the dedicated design and support 
team took on much of the task of course design and development. Design team time 
varied depending on the complexity of the learning activities. We estimate the total 
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costs per MOOC at $203,770 - $325,330. Salary levels at this geographical location may 
be lower than national averages so that costs for the non-teaching personnel could be 
higher on a national average basis, in the order of a few thousand dollars. 

American Museum of Natural History MOOC initiative: resource 
requirements. 

Between September and December 2013, the American Museum of Natural History 
(AMNH) delivered three four-week long MOOCs targeted at science educators. Planning 
efforts began in Spring 2013 and involved a team of museum professionals who had 
significant previous experience in developing and delivering online education. The core 
MOOC production team comprised a project director, a project manager, an in-house 
video producer, an educational technologist, and a senior administrator who also served 
as one of the MOOC instructors. 

Table 5 

Hours Spent by AMNH Personnel to Develop Three MOOCs 


Personnel ingredient 

Hours 

Senior management 

125 

Project director 

454 

Project manager 

980 

Instructors 

910 

Educational technologist 

174 

TA 

400 

Evaluation expert 

16 

Graphic designer 

350 

Video producer 

293 

Video shooter 

63 

Video editor 

210 

HTML writer 

30 

CSS writer 

10 

Legal personnel 

13 

Marketing personnel 

12 

Business manager 

5 

Total for 3 MOOCs 

4, 045 

Hours per MOOC 

1,348 


Sources: Dr. Ro Kinzler, Senior Director, Science Education, AMNH; Dr. Robert Steiner, Director, 
Online Teacher Education Programs, AMNH; Maria Janelli, Senior Manager, Online Teacher 
Education Programs, AMNH. 
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While the museum had already previously developed many digital resources including 
science-content videos and educational essays on science topics, MOOCs presented a 
new challenge to develop lecture-based videos with “talking heads” or voice-over 
PowerPoint presentations, multiple-choice quizzes, peer-graded assessments, and pre- 
and post-course surveys. The personnel effort associated with the production and 
delivery of the three MOOCs are summarized in Table 5, based on time use as logged by 
the AMNH project manager. The project manager and project director spent the 
equivalent of 25 and 11 entire workweeks respectively on the project, while the 
instructors spent, on average, about six workweeks each, shooting videos and 
developing, adapting, or reviewing course content. The core team met once or twice per 
week for one to two hours to plan, design, execute, and review the MOOC production 
and delivery. A TA managed the discussion forums, processed survey responses, and 
reviewed the platform data. 

Using national average salaries and benefits rates for personnel, wherever possible at 
similar positions in postsecondary institutions to allow comparability with the other 
MOOC costs we present, we estimate the personnel costs to develop the three MOOCs 
created by AMNH at $78,470 per MOOC and total costs at $104,620 per MOOC. Of the 
total 39,685 participants who initially enrolled in the three MOOCs, 1,155 completed 
and passed all course requirements. Costs per completer for the MOOCs amount to 
$272. 


Time-by-task and cost analysis for Big Data in Education 
development and delivery. 

Big Data in Education was an eight-week MOOC delivered on the Coursera platform in 
late 2013. Ryan Baker, a faculty member at Teachers College, Columbia University, 
developed the course by adapting a 16-week on-campus version usually taught to classes 
ranging in size from eight to fifteen students. Planning and preparation for the course 
began in mid-March 2013. Big Data in Education was free, open to any participant, and 
non-credit-bearing. There were 48,058 registrants and 526 of them completed the last 
assignment. Baker kept track of time and tasks related to the MOOC in an Excel 
spreadsheet from June (when our study began) to the end of December 2013. Hours 
spent on activities prior to June were estimated. Total time logged plus time estimated 
was 176 hours, with the heaviest burden falling during the first three months of 
planning and preparation of materials, the month prior to launch, and the first few 
weeks of course delivery. Time spent on various tasks included: creating course 
materials such as slides, assignments, and quizzes (58 hours); set-up and video¬ 
recording using ScreenFlow software (46 hours yielding 61/2 hours of finished video 
used in the MOOC); planning, bureaucracy, and coordination with Coursera, the TA, 
and the course production team (37 hours); participating in the forums and responding 
to participant e-mails (26 hours); “debugging” slides, assignments, and quiz questions 
during the course (7 hours); and open office hours (3 hours). 

In addition to Baker, several other personnel worked on the MOOC. A TA spent 
approximately 15 hours per week over 16 weeks for a total of 240 hours. Tasks included 
coordinating among faculty member, video team, and Coursera’s course coordinator; 
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checking that uploaded videos were working; posting assignments and “inline” quiz 
questions (which are embedded in the videos); and participating in the discussion 
forum. Seven individuals from the Educational Data Mining Laboratory at Teachers 
College read and participated in the discussion forums. We estimate two hours per 
person per week over the eight-week period for a total of 112 hours. A senior 
administrator coordinated the production activities one hour per week for eight weeks. 
Two in-house video-specialists edited the video, linked files, requested captioning, and 
uploaded video for 32 hours. A senior educational technologist served as the day-to-day 
project manager for MOOC production and delivery for a total of 75 hours. This 
included monitoring the online discussion forum for technical questions. 

We estimated personnel costs of $29,240 (see Table 6) to replicate the development and 
delivery of Big Data in Education using national average salaries and benefit rates for 
postsecondary personnel at private universities, and total costs of $38,980. With 526 
students completing Big Data in Education, estimated costs per completer are $74. 

Table 6 

Personnel Costs for Big Data in Education Development and Delivery 


Personnel ingredients 

Cost 

Faculty member 

$12,354 

Teaching assistant 

$4,950 

Forum monitors 

$5,377 

Education administrator - educational support 
services 

$584 

Educational technologist: project manager 

$4,725 

Video editor 

$1,248 

Total personnel costs 

$29,238 


Sources: Ryan Baker and Yuan “Elle” Wang, Teachers College, Columbia University; Michael 
Cennamo, CCNMTL, Columbia University. 


Discussion and Conclusions 


Overall, we found that costs of developing and delivering MOOCs at the four institutions 
varied widely, ranging from $38,980 to $325,330 per MOOC. Based on our limited 
sample of eight MOOCs, the key variables in determining costs do not appear to include 
course length or whether the course is designed as a cMOOC or as an xMOOC. Costs 
depend heavily on the number of people involved in the MOOC production process and 
to what extent it is executed “in-house” as opposed to by external professionals. 
Additionally, platform programming costs to facilitate the extensive auto-grading or 
peer-grading functionalities necessary to accommodate the huge enrollments, or to 
provide simulated lab experiences can be high. Course design and delivery has shifted 
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from a solo endeavor to a team effort, often including administrators in offices of digital 
technology, instructional designers, instructional technologists, videographers, and 
project managers. While involvement of multiple professionals is typical of what Bates 
(2005) describes as the “project management” model for web-based course 
development, the higher visibility of MOOCs, and the objective of building or enhancing 
brand appears to have led institutions to dedicate more resources for the planning and 
production of MOOCs compared with regular online courses, often including senior 
level administrators and external video producers who provide very high production 
values. Faculty members are generally undercompensated for the opportunity costs of 
their time to develop MOOC content. 

Cost Comparison: MOOCs, Online, and Hybrid learning 

We did not find pre-existing estimates of MOOC production and delivery costs derived 
from records of personnel effort with which to compare our findings. The E-Learning 
Working Group at the University of Ottawa estimated costs of developing a Coursera 
MOOC at C$110,000 and costs of delivery at C$29,000, for a total of C$139,000 (Boddy 
et al., 2013). The U.S. dollar equivalent of $127,500 falls within the range of our own 
estimates. To provide another point of comparison for our results, we replicated the 
projected costs for Georgia Institute of Technology’s Online M.S. in Computer Science 
program (see GTRC/Udacity, 2013), added a conservative estimate of costs for the head 
TAs/course developers which appear to have been omitted, and calculated an average 
cost per course of $226,ooo-$284,ooo, including both new courses and re-runs. While 
at the high end of our range of cost estimates, these courses provide significantly more 
student support and ongoing instructor involvement. 

Limited publicly available information exists on the institutional costs of contemporary 
postsecondary online courses against which we can compare the costs of MOOCs. Bates 
provides a useful benchmark estimating costs of $35,ooo-$5C>,ooo to develop a regular 
three-credit online course delivered on a learning management system. He notes that, 
within the context of a program, these costs constitute less than 20% of the total, once 
costs of delivery, including student support and assessment, are included (A. Bates, 
personal communications, April 29, 2014, May 15, 2014; Bates & Sangra, 2011). 
Conversely, we estimated that for Big Data in Education the delivery costs constituted 
only 2096-30% of the total cost, with production costs accounting for the majority. Using 
Bates’ guideline, total costs per regular online course for both development and delivery 
would amount to $i75,ooo-$250,ooo, at the higher end of the range we found for total 
MOOC costs. 

Ithaka S+R (2014) attempted to estimate costs of hybrid courses developed and 
delivered at the University System of Maryland. The report indicates that 12 faculty 
members spent between 40 and 506 hours to plan their hybrid courses, some of which 
incorporated MOOCs or MOOC components, plus another four hours per week on 
delivery. If we assume 16-week courses and national average salary and benefits rates 
for average faculty at public universities, the faculty costs amount to between $6,500 
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and $36,000. These numbers fall within the range of our estimates of faculty costs for 
MOOC production and delivery. 

One metric for assessing cost-effectiveness of MOOCs relative to regular online courses 
is institutional cost per student completing the course. In our study we were able to 
estimate this metric in the cases where completion data were available. Cost per 
completer for Big Data in Education was $74 and the average cost per completer across 
the three AMNH MOOCs was $272. By comparison, if we use Bates’ cost estimates for 
regular online courses and spread the total course costs over a typical online class size of 
30 students, cost per completer would be much higher: assuming a completion rate of 
82% for online courses (based on Xu & Jaggars, 2011) cost per completer would be 
$7,ooo-$io,ooo. In practice, cost per completer would be lower if the course is offered 
multiple times, but this is true for both the regular courses and for the MOOC. At a cost 
of $175,000, the number of students completing a regular online course would need to 
reach over 2,300 to be as cost-effective for completion as Big Data in Education. 

It therefore appears that while MOOC production is often more costly than the 
development of regular online courses, the ability to scale MOOCs and the absence of 
associated student supports results in a dramatically lower cost per completer. 
Considering that MOOCs can help achieve other objectives not generally addressed by 
regular online courses, including branding, global reach, and large scale research, 
MOOCs would appear to be a wise use of resources, if only the costs could be recovered 
through tuition or other fees. 

However, it is arguable that course completion per se is not a satisfactory measure of 
effectiveness and that MOOCs should be judged on the quality and quantity of learning 
that takes place. To date, almost no peer-reviewed studies have been published 
comparing pedagogical effectiveness of MOOCs with alternative delivery modes. One 
exception is Colvin et al. (2014) who rigorously document absolute and relative learning 
in a physics MOOC using pre- and post-testing and item response theory, and compare 
the results with on-campus instruction. Colvin et al. find that participants in the MOOC 
showed learning gains slightly higher than for students in a traditional on-campus 
course, but lower than for students in courses that rely on interactive engagement 
pedagogy. As no cost estimates are available in this study, it is not possible to assess 
cost-effectiveness of the MOOC except to note that, given apparently similar learning 
gains, even if the MOOC is more expensive to produce than the on-campus course, its 
ability to serve many more students will likely render it more cost-effective. One 
important caveat is that, with few instructor-student interactions and student supports, 
MOOCs are likely completed only by self-sufficient, motivated students. It is possible 
that MOOCs are cost-effective for this subset of learners, but not for less motivated 
learners. 

Sustainability of MOOCs 

We found that the costs of re-running Connectivism and Connected Knowledge were 
around 38% lower than the costs of the initial offering. Given the intense level of 
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instructor involvement in cMOOCs, this is unlikely to be a useful predictor for xMOOC 
re-runs where instructor involvement may be minimal or absent. One interviewee at a 
community college expected that the re-run costs for the college’s xMOOC would be 
small, perhaps less than $1,000, compared with her estimate of $75,000 for the initial 
offering. Such assumptions should be rigorously tested through careful cost analyses 
and we recommend that, going forward, MOOC producers attempt to document these 
re-run costs to help assess the sustainability of MOOC production. 

Given the highly labor-intensive nature of the process, we do not expect the costs of new 
MOOC production to fall significantly over time. While it appears that revenue streams 
for MOOCs are slowly building, we expect that unless MOOC producers can offer 
credentials of economic value in order to attract fee-paying participants, or can use 
MOOCs to replace traditional offerings more efficiently, most likely by reducing 
expensive personnel, they will not be able to afford ongoing participation in the current 
MOOC experimentation. Free, non-credit bearing MOOCs are likely to remain available 
only from the wealthiest institutions that can subsidize the costs from other sources of 
funds. 

Future Directions 

Several questions remain to be explored with respect to MOOC costs and cost- 
effectiveness and whether they can eventually contribute to reducing the costs of higher 
education. Cost analyses of MOOC re-runs would help ascertain whether costs of re¬ 
offering a MOOC diminish substantially as compared with the initial offering. We 
recommend that future analyses of MOOC costs aim to estimate actual costs of 
materials, equipment, facilities, and overhead as opposed to simply assuming, as we did, 
that these items account for 25% of total costs. Jones (2004), Bates (2005), and Rumble 
(1997)) while acknowledging the difficulty of estimating overhead costs for technology- 
mediated distance instruction, offer valuable guidelines for this endeavor. The feasibility 
of sharing courses across multiple campuses must be explored, as should the question of 
whether, over the longer term, variable costs of MOOCs can be contained by automating 
functions and substituting instructional support provided by expensive faculty members 
with less costly TAs, part-time instructors, or peer-to-peer learning and assessment. 

Studies of MOOC effectiveness with respect to educational outcomes should be 
combined with cost analyses to help determine whether spending more on MOOC 
production and delivery leads to better learning outcomes. For example, does higher 
quality video production lead to higher rates of course completion or greater acquisition 
and retention of knowledge? Does substituting tenured faculty members with non- 
tenured instructors or TA’s affect student performance and learning in MOOCs? While 
it is difficult to set up true experiments in higher education (Bowen, Chingos, Lack, & 
Nygren, 2012), it may be possible to address some of these questions by conducting 
side-by-side comparisons similar to those Ithaka S+R (2014) executed at the University 
System of Maryland. 
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To answer the question of whether MOOCs are a cost-effective means to deliver 
education, we must be able to compare the costs of MOOCs to the costs of alternative 
delivery mechanisms, as well as the effectiveness of each alternative with respect to a 
common outcome of interest, such as increasing participants’ level of knowledge or skill 
in a specific subject area. Generating cost-effectiveness ratios for a number of 
educational alternatives including MOOCs would allow decision-makers to choose 
which programs represent the best investments of resources. Longitudinal studies 
tracking post-MOOC outcomes such as sequences of courses taken, professional 
certifications obtained, or job opportunities received would help assess the longer term 
economic value of participating in these courses and allow for cost-benefit analyses to 
estimate the overall returns to society of investing in MOOC creation. 
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